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DETAILED ACTION 

Claim Rejections - 35 USC § 103 

1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

2. Claims 1, 2, 5, and 7-10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Olympus (Japanese Publication Number 06-070267, pub Mar 11, 1994) in view of Petrich (US 
Pub 2006/0280368 Al). 

As per claim 1, Olympus teaches the claimed: 

1 . A three-dimensional image display method (in figure 2 and in the English translation of the 
abstract, where it states that the device produces image data to be displayed) comprising: 

"detecting directions of incident light emitted" and "at a plurality of detectors" (in figure 3, the 
photodetectors 11, and [0008] of the English Translation, where two or more photodetectors 
analyze and determine the direction of lighting and [0008] of the English Translation where it 
states the photodetectors are used to detect the direction of indoor lighting, thus the system in a 
room indoors produces a shadow because the system can detect the light, i.e. in figures 4-6); 



Application/Control Number: 1 0/6 12,009 Page 3 

Art Unit: 2628 

"comparing each of the positions of the light source and a virtual position of a display object in a 
three dimensional image displayed in real space to obtain a shadow for applying to the display 
object from directions of the light sources"; ([0006] and [0008] of the English Translation which 
teach of generating the image in figure 2b according to the detected light direction and shading 
their respective synthetic image according to the detected lighting direction so that the synthetic 
image generated is in agreement with the detected light direction); 

displaying the three-dimensional image with the shadow (in figure 2 and in the English 
translation of paragraph [0006]). 

Olympus does not explicitly teach the remaining claim limitations. 
Petrich teaches the claimed: 

calculating positions of the plurality of light sources existing in real space based on the detected 
directions ([0004], "The spherical dome center of the target device provides a visual reference 
indicator as to direction of one or more light sources in much the same way the position of the 
sun may be ascertained from viewing the Earth's moon " and [0036], "Mathematical algorithms 
may also be used in place of a know table of values to calculate the position oflisht "). 

In addition, Olympus does not teach the claimed: 
"a plurality of light sources" and 
"the shadow being caused by the light sources". 
Petrich teaches the claimed: 
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"a plurality of light sources" and "the shadow being caused by the light sources" ([0004], "The 
spherical dome center of the target device provides a visual reference indicator as to direction of 
one or more light sources in much the same way the position of the sun may be ascertained from 
viewing the Earth's moon ", [0005], "to determine lighting and shadow conditions ", and [0003] 
"so that each processed photograph may be merged with other processed photographs or three- 
dimensional computer rendered images having similar attribute values to form a realistic 
montage"). 

It would have been obvious to one of ordinary skill in the art at the time of invention to combine 
Olympus with Petrich in order to obtain more accurate information relating to the light sources 
such as a light position in addition to the light direction. Olympus is modified by Petrich by 
incorporating the position calculation technique as mentioned in [0036] of Petrich into the 
shading processing unit 13 in figure 1 of Olympus. Furthermore, Olympus would perform 
correctly when used with the plurality of light sources of Petrich. For example, if two light 
sources, two light bulbs, from the same indoor light fixture are used, the system of Olympus 
would correctly shadow the object because the two light sources are located near each other in 
the room. 

As per claim 2, Olympus does not explicitly teach the claimed limitations. 
Petrich teaches the claimed: 

2. The method according to claim 1, further comprising: 
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detecting lightness of the light sources at the detectors ([0005], "Another aspect of the present 
invention is a process of digitally analyzing photographic images with specialized software to 
determine lighting and shadow conditions such as a vertical and horizontal direction of light 
sources in relation to a target device, degree of light diffusion, degree of ambient illumination, 
and light color"). 

It would have been obvious to one of ordinary skill in the art at the time of invention to detect 
lightness for a plurality of sources as taught by Petrich with the teachings of Olympus in order to 
gather more accurate information relating to the surround environment, which in turn, the data is 
used to produce more realistic synthetic images. 

As per claim 5, this claim is similar in scope to claim 1, and thus is rejected under the same 
rationale. 

As per claim 7, Olympus teaches the claimed: 

7. The device according to claim 5, further comprising: a display surface configured to display 
the three-dimensional image, wherein: the direction detectors are disposed on at least one of the 
display surface and a surface adjacent to the display surface, (in figure 1 where the detectors 11 
are both on a display surface and adjacent to the display surface where these detectors are part 
of the head mount display where the display surface is the portion that covers the eyes of a user 
when wearing the device). 



As per claim 8, Olympus teaches the claimed: 
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8. The device according to claim 5, further comprising: a display surface configured to display 
the three-dimensional image, wherein: the direction detectors are disposed to be adjacent to the 
display surface (in figure 1 where the detectors 11 are adjacent to the display surface which is 
the part of the head mounted display where the display surface is the portion that covers the eyes 
of a user when wearing the device). 

As per claim 9, Olympus teaches the claimed: 

9. The device according to claim 5, wherein the direction detectors are disposed at a position 
where the direction detectors detect the light emitted from the light sources located in the same 
direction as at least one of a display direction of the three dimensional image and a direction in 
which the three-dimensional image is observed (in figure 1 where it shows detectors 11 are 
disposed at a direction the same as a display direction of the head-mounted display in relation to 
the user). 

As per claim 10, Olympus does not explicitly teach the claimed limitations. 
Petrich teaches the claimed: 

10. The device according to claim 5, wherein: each of the direction detectors includes a three- 
primary colors detection unit that adds colors to the shade. ([0030], "other methods of detecting 
said target devices within a digital image include the use of: HSL, HSV, LAB, RGB and other 
digital image color spaces " where RGB is a red, green, and blue color sequence). 
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It would have been obvious to one of ordinary skill in the art to use the color detectors as taught 
by Petrich with Olympus in order to more accurate capture color data for analysis which are used 
to produce better synthetic images. 

3. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Olympus 
(Japanese Publication Number 06-070267, pub Mar 11, 1994) in view of Petrich (US Pub 
2006/0280368 Al) in view of Drettakis et al. (NPL Document "Interactive Common 
Illumination for Computer Augmented Reality", herein referred to as "Drettakis"). 

As per claim 4, Olympus does not explicitly teach the claimed limitations. 
Drettakis teaches the claimed: 

obtaining a position of a single virtual light source, which represents the plurality of light 
sources; (at the top of the page where section 6.3 starts, "In addition, special attention must be 
taken in the re-scaling of the image before display since the addition of a source can add an 
order (or orders) of magnitude to the radiosity values of the scene ". Here, the accumulation of 
radiosity represents a single virtual light source. The reference refers to "adding" light onto the 
total radiosity for each light source. When every light source is considered, one can represent 
this radiosity as a single virtual light source through the addition of all individual light sources) 

comparing the position of the virtual light source and the virtual position of the display object in 
the three-dimensional image to obtain a virtual shadow for applying to the display object from a 
direction of the single virtual light source, the virtual shadow being caused by the single virtual 
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light source (in figures 3 and 4. Figure 4 shows the result of the comparison between the virtual 
light source (as calculated as the accumulated radiosity in figure 3) and the virtual position of 
the display object (the floating box above the desk). The comparison of the positions and 
relative positional relationship result in the rendering of a soft and varying shadow on the desk 
below the floating object. The varying nature of this shadow is due to the plurality of light 
directional components and is represented by the virtual light source). 

It would have been obvious to one of ordinary skill in the art at the time of invention to combine 
Olympus, Petrich, and Drettakis. Drettakis teaches one advantage of the combination (in the 
abstract, "Our new framework will hopefully lead to CAR systems with interactive common 
illumination without restrictions on the movement of real or synthetic objects, lights and 
cameras" where the combining of light sources in Drettakis helps accomplish this stated goal by 
making it easier to do interactive illumination without restrictions). Olympus and Drettakis is 
modified by Drettakis by incorporating the radiosity values and virtual light source of Drettakis 
into the shading calculations as performed in the shading processing unit 13 in figure 1 of 
Olympus. 

Claim 16 is rejected under 35 U.S. C. 103(a) as being unpatentable over Olympus (Japanese 
Publication Number 06-070267, pub Mar 11, 1994) in view of Aubrey (US Patent 6,614,427). 



As per claim 16, Olympus teaches the claimed: 
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detecting a relative position of a light source existing in real space (in figure 4 where the relative 
position of the light source is determined by the shadow detected) based on a display surface (in 
figure 2B where the display output is shown and in figure 1 where the display surface is the visor 
11); 

comparing the relative position of the light source and a relative position of a three-dimensional 
image displayed in real space based on the display surface and 

to obtain a relative positional relation therebetween ([0006] and [0008] of the English 
Translation which teach of generating the image in figure 2b according to the detected light 
direction and shading their respective synthetic image according to the detected lighting 
direction. The correct shadowing in figure 2B requires the knowledge of the relative positional 
relationship between the light source and display image); 
Olympus does not explicitly teach the remaining claim limitations. 

Aubrey teaches the claimed: 

a display surface as a reference plane (col 5, lines 44-52, "The viewing situation described above 
is appropriate for images of modest size. Other stereoscopic viewing techniques may be 
employed, including: ... any of several stereoscopic CRT techniques, including alternating field 
arid lenticular autostereoscopy; holography; and lenticular stereography". In this instance, the 
display surface is CRT display. The CRT display surface acts a reference plane). 
shading the three-dimensional image, wherein the three-dimensional image is projected in real 
space such that the three-dimensional image spatially spreads out in real space (in figure 7 where 
the cup is a three-dimension image projected in real space from a plane, the floor and col 5, 
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lines 44-52, "The viewing situation described above is appropriate for images of modest size. 
Other stereoscopic viewing techniques may be employed, including: ... any of several 
stereoscopic CRT techniques, including alternating field arid lenticular autostereoscopy; 
holography; and lenticular stereography "). 

It would have been obvious to one of ordinary skill in the art at the time of invention to combine 
Olympus and Aubrey in order to generate a more interesting and useful display output. The 
output of Aubrey provides a more impressive three-dimensional display. Olympus is modified 
by Aubrey by replacing the head mounted display (and its associated reference plane) of 
Olympus with the holographic-like display used in Aubrey in figure 7. In this instance, the 
reference plane of Aubrey (the floor in figure 7) would replace the reference plane in the head 
mounted display in Olympus. 

Claim 17 is rejected under 35 U.S. C. 103(a) as being unpatentable over Olympus (Japanese 
Publication Number 06-070267, pub Mar 11, 1994) in view of Petrich (US Pub 2006/0280368 
Al) in further view of Aubrey (US Patent 6,614,427). 

As per claim 17, this claim is similar in scope to claim 16, and thus is rejected under the same 
rationale. 

It would have been obvious to one of ordinary skill in the art at the time of invention to combine 
Olympus, Petrich, and Aubrey. The motivation of claim 1 6 is incorporated herein. 



Response to Arguments 



Application/Control Number: 10/612,009 
Art Unit: 2628 

1 . Applicant's arguments filed 21 1 5/2008 
persuasive. 
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been fully considered but they are not 



Applicant argues: 

The Office Action asserts that because Olympus discloses indoor lighting, this indoor lighting 
may include a hypothetical room with two lights. This creation by the Examiner of a hypothetical 
two-light room is clear evidence that Olympus does not disclose a plurality of light sources . . . 
Thus, Olympus appears to disclose detecting a single direction of a light source, rather than a 
"plurality of light sources," as required by claims 1 and 5. Accordingly, the Office Action fails to 
meet the requisite burden of proof to establish disclosure or suggestion of at least this claimed 
feature (page 8 in filed response). 

The examiner respectfully maintains that the rejections are proper because the prior art 
combination of both Olympus and Petrich teach the claimed features as argued by applicant. 

For example, the reference of Olympus says nothing of limiting its application to only 1 
light source. In addition, Petrich teaches of using a plurality of light sources in paragraph 
[0004], "The spherical dome center of the target device provides a visual reference indicator as 
to direction of one or more light sources in much the same way the position of the sun may be 
ascertained from viewing the Earth's moon". Through the combination as presented in claim 1 of 
this office action, the combination of Olympus and Petrich teach the claimed features. For 
example, Olympus teaches of performing shadow rendering on virtual objects based upon detect 
light from a single direction (see figures 1 and 2). Petrich teaches of detecting light positions 
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from a plurality of light sources (see paragraphs [0004] and [0036]). By combining these two 
references, Olympus can perform the shadow rendering using a plurality of light sources. 

In addition, one of ordinary skill in the art would expect a reasonable expectation of 
success through the combination of references. The photo-sensors in figure 4-6 of Olympus will 
perform equally well with 1 light source or 2 light sources located near each other, i.e. two bulbs 
on the same indoor light fixture. In this case, the 2 light sources of the lighting may originate 
from similar directions, and thus the shading computations performed in Olympus would 
perform equally well with only 1 light source or 2 light sources near each other. In addition, 
since the light sources are in similar directions the light sensor and shading calculations would be 
correct in Olympus. Thus, claim 1 of applicant's invention appears not to be novel because it is 
appears to be an obvious variation of the system of Olympus. 

Applicant's remaining arguments have also been considered but are moot in view of the new 
ground(s) of rejection. 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
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the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Daniel F. Hajnik whose telephone number is (571) 272-7642. 
The examiner can normally be reached on Mon-Fri (8:30A-5:00P). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka J. Chauhan can be reached on (571) 272-7782. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Ulka Chauhan/ 

Supervisory Patent Examiner, Art Unit 2628 
DFH 



